The growth of the Internet and the popularity of interactive entertainment such as video games have led to opportunities for customization of assets within the context of video game programs and other programs for interactive entertainment. Video games allow a user to interact with a simulated terrain and other computer generated objects. Often, the user's interaction is by means of a computer generated symbol or character known as an avatar. Avatars and simulated terrain are often generated by software developers. There have been several attempts at customization of avatars and terrain features. These attempts are often limited by the capabilities of the interface controller that is used in conjunction with the game. Often, the interface has a few buttons and joysticks. These controls can be awkward to use for tasks such as drawing game characters or terrain features.
Recently, Microsoft Corporation developed an interface built around a digital camera. The camera captures a digital image of a scene and specialized software generates a height field from an analysis the digital image. Certain implementations use a depth sensing camera to capture depth information from the digital image.
The height field may then be used to generate terrain features for a video game. Unfortunately, this technique generates a single height field and thus cannot separate different objects from the image.
Another prior art interface uses a regular camera, and infers depth based on the shading of the image. Such an approach may construct a height field that is somehow correlated to the brightness of the image. For example, shadows would be low, while bright areas would be high. Another developer has used a depth camera to segment objects out of a scene and to insert synthetic objects into a real scene. This interface is described in detail at http://www.ee.ic.ac.uk/gelenbe/index_files/Games.pdf.
Unfortunately, cameras that detect depth information tend to be expensive and incorporating depth information into a simulated environment using such prior art techniques can be complicated.
It is within this context that embodiments of the invention arise.